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1.  INTRODUCTION 

It  appears  to  be  the  rule  rather  than  the  exception  that  usefully  detailed  stochastic 
models  for  complex  systems  are  such  that  it  is  extremely  difficult  or  impossible  to 
obtain  an  exact  analytic  solution.  Simulation  is  essentially  a  controlled  statistical 
sampling  technique  which  can  be  used  to  study  complex  stochastic  systems  when 
analytic  and/or  numerical  techniques  do  not  suffice.  We  concentrate  here  on 
discrete-event  digital  simulation  in  which  the  behavior  of  a  specified  stochastic  system 
is  observed  by  sampling  on  a  digital  computer  system  and  stochastic  state  transitions 
occur  only  at  a  set  of  increasing  (random)  epochs  of  time.  In  discrete-event  simulations 
most  of  the  stochastic  processes  that  we  encounter  have  piecewise-constant  sample 
paths. 

When  simulating,  we  experiment  with  a  stochastic  system  and  observe  its 
behavior.  In  the  course  of  the  simulation  we  measure  certain  quantities  associated  with 
the  system,  and  using  statistical  techniques,  draw  inferences  about  characteristics  of 
well  defined  random  variables.  The  most  obvious  methodological  advantage  of 
simulation  is  that  in  principle  it  is  applicable  to  stochastic  systems  of  arbitrary 
complexity.  It  is,  however,  a  decidedly  nontrivial  matter  in  practice  to  obtain  from  a 
simulation  information  which  is  both  useful  and  accurate,  and  to  obtain  it  at  reasonable 
cost.  The  difficulties  arise  primarily  from  the  inherent  variability  in  a  stochastic 
system,  and  it  is  necessary  to  seek  theoretically  sound  and  computationally  efficient 
methods  for  carrying  out  the  simulation.  Apart  from  implementation  considerations, 
important  concerns  for  simulation  relate  to  generation  methods  for  sample  paths  of  the 
stochastic  system  under  study,  the  design  of  simulation  experiments,  and  the  analysis  of 
simulation  output.  Since  results  of  a  simulation  are  based  on  observation  of  a  stochastic 
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system,  it  is  absolutely  essential  that  some  assessment  of  the  precision  of  results  be 
provided. 

Implicit  in  the  implementation  of  any  simulation  is  the  definition  of  an 
appropriate  "state”  for  the  system.  Heuristically,  the  system  state  maintains  sufficient 
information  about  the  system  so  that  state  transitions  that  occur  over  time  completely 
determine  the  quantities  of  interest.  This  "state  of  the  system  at  time  /"  constitutes  a 
stochastic  process  in  continuous  or  discrete  time.  When  carrying  out  the  simulation  we 
observe  the  behavior  of  this  process  as  it  evolves  in  time.  In  order  to  do  so  it  is 
necessary  to  have  a  means  of  generating  sample  paths  of  this  process  and  to  have 
methods  for  obtaining  valid  estimates  of  the  quantities  of  interest  in  the  system. 

In  this  paper  we  focus  on  simulation  methods  for  non-Markovian  systems  in 
continuous  time;  i.e.,  systems  whose  state  cannot  be  modeled  as  a  Markov  chain  with 
countable  state  space.  This  is  characteristic  of  local  area  computer  network  models  (see 
e.g.  Loucks,  Hamacher,  and  Preiss  [9])  where  it  is  important  to  incorporate  system 
control  algorithms  explicitly  into  the  simulation  model.  We  restrict  attention  to  those 
discrete-event  simulations  whose  underlying  stochastic  process  can  be  represented  as  a 
generalized  semi-Markov  process  (gsmp)  in  the  sense  of  Whitt  [15]. 

In  addition  to  providing  a  framework  for  generating  sample  paths  of  the 
underlying  stochastic  process  of  the  simulation,  the  gsmp  representation  is  particularly 
useful  for  simulation  of  non-Markovian  systems  in  that  it  leads  to  methods  for 
obtaining  recurrence  properties  of  the  underlying  stochastic  process;  cf.  Glynn  [5]. 
Such  properties  are  needed  to  establish  estimation  procedures  based  on  regenerative 
processes;  cf.  Fossett  [4]  and  Iglehart  and  Shedler  [6].  For  specific  non-Markovian 
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systems  (e.g.,  ring  and  bus  network  models)  it  can  be  difficult  to  determine  conditions 
(distributional  assumptions)  under  which  the  underlying  stochastic  process  is 
regenerative.  In  this  paper  we  develop  "geometric  trials"  arguments  (cf.  Nummelin 
[11]  and  Tuominen  and  Tweedie  [14])  which  can  be  used  to  show  the  applicability  of 
regenerative  simulation  methods. 

2.  GENERALIZED  SExMI-MARKOV  PROCESSES 

Heuristically,  a  gsmp  (Matthes  [10],  Konig,  Matthes  and  Nawrotzki  [7],  [8]) 
moves  from  state  to  state  in  accordance  with  the  occurrence  of  events  associated  with 
the  occupied  state.  Each  of  the  several  possible  events  associated  with  a  state  compete 
to  trigger  the  next  transition,  and  each  of  these  events  has  its  own  distribution  for 
determining  the  next  state.  At  each  state  transition  of  the  gsmp,  new  events  may  be 
scheduled.  For  each  of  these  new  events,  a  clock  indicating  the  time  until  the  event  is 
scheduled  to  occur  is  set  according  to  an  independent  (stochastic)  mechanism.  If  a 
scheduled  event  does  not  trigger  a  transition  but  is  associated  with  the  next  state,  its 
clock  continues  to  run;  if  such  an  event  is  not  associated  with  the  next  state,  it  is 
abandoned. 

Following  Whitt  [15],  formal  definition  of  a  gsmp  is  in  terms  of  a  general  state 
space  Markov  chain  (gssmc)  which  describes  the  process  at  successive  epochs  of  state 
transition.  Let  5  be  a  finite  or  countable  set  of  states  and  £  =  be  a  finite 

set  of  events.  For  se  S,  E(s)  denotes  the  set  of  all  events  that  can  occur  when  the  gsmp 
is  in  state  s.  When  the  process  is  in  state  s,  the  occurrence  of  an  event  e€  E(s)  triggers 
a  transition  to  a  state  s' .  We  denote  by  p(.s';s,e )  the  probability  that  the  new  state  is  *' 
given  that  event  e  triggers  a  transition  in  state  s.  For  each  se  S  and  e€  E(s)  we  assume 
that  p(  \s,e)  is  a  probability  mass  function.  The  actual  event  ee  E{s)  which  triggers  a 


4 


transition  in  state  s  depends  on  docks  associated  with  the  events  in  E(s)  and  the  speeds 
at  which  these  clocks  run.  Each  such  clock  records  the  remaining  time  until  the  event 
triggers  a  state  transition.  We  denote  by  rfj  (*0)  the  deterministic  rate  at  which  the 
clock  cjf  associated  with  event  ejt  runs  in  state  s;  for  each  seS,  rfi  =  0  if  e E{s).  We 
assume  that  rfj> 0  for  some  e,  €  E(s).  (Typically  in  applications,  all  speeds  rsj  are  equal 
to  one.  There  are,  however,  models  in  which  speeds  other  than  unity  as  well  as 
state-dependent  speeds  are  convenient.  For  example,  zero  speeds  are  needed  in 
queueing  systems  with  service  interruptions  of  the  preemptive-resume  type;  cf.  Shedler 
and  Southard  [12].) 

For  s£  S  define 

C(5)  =  {(cj,...,^):  Cj  i  0  and  c,  >  0  if  and  only  if  £(j); 

(2.1)  cir'si*cjrsj  for  '*J  with  cicjrsirsj  >  °1- 

The  conditions  in  Equation  (2.1)  ensure  that  no  two  events  simultaneously  trigger  a 
transition  (as  defined  below).  The  set  C(j)  is  the  set  of  possible  clock  readings  in  state 
s.  The  clock  c(  and  event  e are  said  to  be  active  in  state  s  if  e;  ££($).  For 
se  S  and  c£  C(j),  let 

(2.2)  /*  =  t’(j,c)  =  min  {c, r’J], 

{i:e,  £  £($)} 

« 

where  c,rj*  is  taken  to  be  +  »  when  rsi  =*  0.  Also  set 

(2.3)  Cf  «  c‘(s,c )  =  Cj  -  t*(s,c)rsi,  e,  £E(s) 

and 

(2.4)  i*  »  i’(s,c)  *  i  such  that  e,  £  E{s)  and  c’(s,c)  =  0. 


.4 


Beginning  in  state  s  with  clock  vector  c,  /’(j,c)  is  the  time  to  the  next  state  transition 
and  /*(r,c)  is  the  index  of  the  unique  triggering  event  e  =  e*(i,e)  =  e.*^  cy 

At  a  transition  from  state  s  to  state  s'  triggered  by  event  e*,  new  clock  times  are 

I  |  V  I  0 

generated  for  each  e  £  N(s  ;s,e  )  =  £(r  )-(£(j)-{e  }).  The  distribution  function  of  such 
a  new  clock  time  is  denoted  by  F(-;s  ,e  ,s,e*)  and  we  assume  that  £(0;r',e',r,e*)  =  0. 
For  e  £  0(s\s,e’)  =  £(r')fl  (£($)-{«*}),  the  old  clock  reading  is  kept  after  the 
transition.  For  e  £  (£(*)-{/})-£(/),  event  e  ceases  to  be  scheduled  after  the 
transition. 

Next  consider  a  gssmc  {(£J|,Cn):«^0}  having  state  space 

2  =  U  (W*C(r)) 

s£S 

and  representing  the  state  ( Sn )  and  vector  (C„)  of  clock  readings  at  successive  state 
transition  epochs.  (The  /th  coordinate  of  the  vector  Cn  is  denoted  by  C„ ;-  )  The 
transition  kernel  of  the  Markov  chain  {(S'n,Cn):/iaO}  is 

(2.5)  f»((r,c)*4)  « p(s';s,em)  £(<*,;/, efry,e*)  1[0  <ij(c,). 

etems')  e,  £  <?(•»') 

where  N(s  )  »  N(s';s,e’),  O(j')  -  0(i';s,e’),  and 

A  »  {i'}x{(c'1,...,c^)  £  C(j'):  c]  £  a,  for  *,££(*')}. 

The  set  A  is  the  subset  of  2  which  corresponds  to  the  gsmp  entering  state  s'  with  the 
reading  cf  on  the  clock  associated  with  event  et  £  £(r  )  set  to  a  value  in  [0,af]. 


Finally,  the  gsmp  is  a  piece-wise  constant  continuous  time  process  constructed 
from  the  gssmc  {(.?„, Cn):/iaOj  in  the  following  manner.  Denote  by  the  time  of  the 
nth  state  transition,  n^O,  (We  assume  that 

/’{sup  =  +  «  I  (^q.Cq)}  *  l 
nil 

for  all  initial  states  (SQ, C0).)  Then  set 

(2.6)  XU)  -  SNUy 

where 


NU)  -  max  {n  >  0:  s 

The  process  {X(/):/20}  is  a  gsmp. 

The  following  examples  illustrate  the  use  of  the  gsmp  model  as  a  formal 
specification  of  a  discrete-event  simulation  of  a  non-Markovian  system. 

(2.7)  EXAMPLE.  Consider  a  unidirectional  ring  network  having  a  fixed  number  of 
ports ,  labelled  1,2 in  the  direction  of  signal  propagation.  At  each  port  message 
packets  arrive  according  to  a  random  process  and  queue  externally.  A  single  control 
token  circulates  around  the  ring  from  one  port  to  the  next.  The  time  for  the  token  to 
propagate  from  port  N  to>  port  1  is  a  positive  constant,  R^,  and  the  time  for  the  token 
to  propagate  from  port  j- 1  to  port  j  is  a  positive  constant,  /?  ._ j,  j  =  2,3 When  a 
port  observes  the  token  and  there  is  a  packet  queued  for  transmission,  the  port  converts 
the  token  to  a  connector  and  transmits  a  packet  followed  by  the  token  pattern;  the 
token  continues  to  propagate  if  there  is  no  packet  queued  for  transmission.  By 
destroying  the  connector  prefix  the  port  removes  the  transmitted  packet  when  it 
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returns  around  the  ring.  Assume  that  the  time  for  port  j  to  transmit  a  packet  is  a 
positive  random  variable,  Lj,  with  finite  mean.  Also  assume  that  packets  arrive  at 
individual  ports  randomly  and  independently  of  each  other;  i.e.,  the  time  from  end  of 
transmission  by  port  j  until  the  arrival  of  the  next  packet  for  transmission  by  port  j  is  a 
positive  random  variable,  Aj,  with  finite  mean.  Note  that  there  is  at  most  one  packet 
queued  for  transmission  at  any  time  at  any  particular  port. 

Set 

(2.8)  X(t)  =  (Z^/) . 

where 

1  if  there  is  a  packet  queued  for  transmission  at  port  j  at  time  t 

ZjU)  - 

0  otherwise 

j  if  port  j  is  transmitting  a  packet  at  time  / 

MU)  - 

0  if  no  port  is  transmitting  a  packet  at  time  / 

N(t)  »  1  if  at  time  /  port  N  is  transmitting  a  packet  or  the  token  is  propagating  to  port 
1,  and  N(i)  *  j  if  at  time  t  port  y'-l  is  transmitting  a  packet  or  the  token  is  propagating 
to  port  j,  j  -  2 

The  process  {<?(/) :/20}  defined  by  Equation  (2.8)  is  a  gsmp  with  a  finite  state 
space,  5,  and  event  set,  E.  The  events  in  the  set  E  are:  "observation  of  token,"  "end 
of  transmission,"  and  "arrival  of  packet  for  transmission  by  port  j,"  j  -  1,2,. ..,N.  For 
s  <■  (zj,...,zy,m,«)  €  S,  the  event  sets  E(s )  are  as  follows.  The  event  "end  of 
transmission"  €  E(s)  if  and  only  if  m> 0  and  "observation  of  token"  £  E(s)  if  and  only  if 
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m  =c  0.  The  event  "arrival  of  packet  for  transmission  by  port  j  "  €  E(s)  if  and  only  if 
Zj  —  0  and  m?tj. 

If  s  =  (z1,...,zAr,m,m  +  l)eS  with  0<m<N,  s'  =  (zj,...,z^,0,m  +  1)  (or  if 
s  -  e  S,  s'  =  (z1,...,2jV,0,l))  and  e="end  of  transmission,"  then  the  state 

transition  probability  p(s';s,e)  =*  1.  If  j  =  (z1,...,:Ji_1,l>z(|+],...^,0,/i)£J  with  n<N, 
s'  »  (z1,...,zJ|_1,0,zII+l,...,zAf,n,/i  +  1)  (or  if  *  =  (zl,„.,zJV_1,l,0JV)  £  S  and 
s'  =  (z1,...rz^1,0,Ar,l))  and  e—  "observation  of  token,"  then  />(j';j,e)  =  1.  If 

*  -  <2i . s>  =  . z„,0,n  +  1),  and 

e=" observation  of  token,"  then  p(s';.s,e)  =  1.  If  s  =  (z^,...,z.v0yZj+l,...^N,m,m  +  1  )eS 
with  mjtj  and  0<m<N,  s'  =  (zj,...,z.  j,l,zy+1,...,zAr,m,m  +  1),  (or  if 

s  =*  (zv...jj a,0,Zj+v...,zn,N,1)£  S  with  N*j,  s'  -  (z1,...,zJ..1,l,z;+1 . zA,,#,l)),  and 

e-  "arrival  of  packet  for  transmission  by  port  j,"  then  p(s'\s,e)  =  1.  All  other  state 
transition  probabilities  p(s';s,e )  are  equal  to  zero. 

The  distribution  functions  of  new  clock  times  for  events  e' e  #(/;*,?’)  are  as 

follows.  If  e  =  "end  of  transmission"  and  s'  =  (zx,...,zN,m,n),  then 

F(x;s’  ,e'  ,s,e*)  =■  />{Lnjs;x}  for  all  s  and  e  such  that  p(s  ;s,e')>0.  If  e'  =»  "observation  of 

token"  and  s'  *  (zj,...,z^,0,n),  then  F(x-,s'  ,e  ,s,e*)  =  1^  «)(•*)•  ^  e  =  "arrival  of 

packet  for  transmission  by  port  /'  and  s'  =  ( zI,...„z/._1,0,zy+1,...,zAr,0x/'  +  1),  then 

« 

F(x;sV,s,e*)  =*  P{A}Zx\. 

(2.9)  EXAMPLE.  Consider  a  ring  network  having  a  fixed  number,  K,  of  equal  size 
slots,  and  a  fixed  number  of  equally  spaced  ports,  labelled  1,2,...,#  in  the  direction  of 
signal  propagation.  At  each  port  constant  (slot  size)  length  message  packets  arrive 
according  to  a  random  process  and  queue  externally.  The  propagation  delay  from  one 
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port  to  the  next  is  a  positive  constant,  R.  We  assume  that  the  number  of  ports,  N,  is  a 
multiple  of  K  and  (so  that  there  is  no  loss  of  utilization  due  to  "unused  bits")  that  the 
time  to  transmit  a  message  packet  is  equal  to  NR/K.  The  lead  "full/empty"  bit 
maintains  the  status  of  each  slot.  Subject  to  the  restriction  that  no  port  can  hold  more 
than  one  slot  simultaneously,  a  port  that  has  a  packet  queued  for  transmission  and 
observes  the  status  bit  of  an  empty  slot  sets  the  bit  to  1  ("full")  and  starts 
transmission  (begins  filling  the  slot).  Transmission  ends  when  the  slot  contains  the 
entire  packet.  When  the  status  bit  of  the  filled  slot  propagates  back  to  the  sending 
port,  the  port  resets  the  bit  to  0  ("empty")  and  releases  the  slot.  To  ensure  that  all 
ports  have  an  opportunity  to  transmit,  a  port  which  releases  a  slot  passes  the  empty  slot 
to  the  next  port.  Assume  that  packets  arrive  at  individual  ports  randomly  and 
independently  of  each  other;  i.e.,  the  time  from  end  of  transmission  by  port  j  until  the 
arrival  of  the  next  packet  for  transmission  by  port  j  is  a  positive  random  variable.  A,, 
with  finite  mean.  Note  that  there  is  at  most  one  packet  queued  for  transmission  at  any 
time  at  any  particular  port. 


0  otherwise 


10 


for  i  =*  1,2 


j  if  port  j  holds  slot  ;  ?.♦.  time  t 

M,U)  = 

0  otherwise 

.Vf(/)  =  1  if  at  time  t  the  status  bit  of  slot  /  is  propagating  to  port  1,  and  .V((/)  =  j  if  at 
time  i  the  status  bit  of  slot  i  is  propagating  to  port  j,  j  =  2,3,. ..,N.  For  any  i  (1  iizK) 
the  vector  (Zx(t),...,ZNU),Ml(t),...,MK(0,Nj{t))  contains  the  same  information  about 
the  system  as  the  vector  X(t).  Incorporation  of  all  the  components  N j  (/),.. .,Aa(0  into 
the  state  vector  facilitates  generation  of  the  process. 

The  process  defined  by  Equation  (2.10)  is  a  gsmp  with  a  finite  state 

space,  5,  and  event  set,  E.  The  events  in  the  set  £  are:  "observation  of  status  bits  by 

ports"  and  "arrival  of  packet  for  transmission  by  port  j,"  j  =  1,2 Let 

s  =  (z1,..MzJV,/n1 mK,nv...,nK)e  S.  The  event  "observation  of  status  bits  by 

ports"  €E(s)  for  all  seS.  The  event  "arrival  of  messsage  for  transmission  by  port 

j"eE(s )  if  and  only  if  Zj  -  0  and  for  each  i  either  (i)  m^j  or  (ii)  =  j  and 

rtj-1  =  j- 1  +  /  (mod  AO  for  some  integer  /  such  that  N/K<lzN.  Note  that  the  ends  of 

transmission  coincide  with  the  occurrence  of  particular  "observation  of  status  bits  by 

ports"  events.  Suppose,  for  example,  that  there  are  N  =  4  ports  and  K  —  2  slots  and 

that  s  -  (0,0,0,0,1, 0,2,4);  i.e.,  port  1  is  transmitting  a  packet  in  slot  1,  slot  2  is  empty, 

% 

the  status  bit  of  port  1  is  propagating  to  port  3,  and  the  status  bit  of  slot  2  is 
propagating  to  port  4.  Then  the  occurrence  of  the  event  "observation  of  status  bits  by 
ports"  in  state  s  corresponds  to  an  end  of  transmission  by  port  1. 

(2.11)  EXAMPLE.  Consider  a  collision-free  bus  network  (cf.  Eswaran,  Hamacher,  and 
Shedler  [3])  with  N ports,  numbered  from  left  to  right;  see  Figure  1.  Message 
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packet  traffic  on  the  passive  bilateral  bus  is  transmitted/received  by  port  j  at  tap  BO)- 
In  addition  to  the  bus,  a  one-way  logic  control  wire  also  links  the  ports.  Associated 


with  each  port  j  is  a  flip-flop,  S (J),  called  the  send  flip-flop.  The  signal  PO).  called  the 
OR -signal,  tapped  at  the  control  wire  input  to  port  j  is  the  inclusive  OR  of  the  send 
flip-flops  of  all  ports  to  the  left  of  port  j.  Denote  by  T  the  end-to-end  bus  propagation 
delay.  [For  technical  reasons,  T  actually  must  be  the  end-to-end  propagation  delay  plus 
a  small  (fixed)  quantity.]  Denote  the  actual  propagation  delay  along  the  bus  between 
port  i  and  port  j  by  T(iJ),  ij  =  1,2,...,//.  Thus,  T(iJ)  =  T(J,i)<T  for  all  ij  and 
T(iJ)  +  T(J,k )  =  T(i,k )  for  all  i<j<k.  (We  assume  that  T(iJ)jtT(kJ)  for  distinct  i,k 
and  all  j.)  Let  R(J )  be  the  propagation  delay  (including  gate  delays)  along  the  control 
wire  from  port  j  to  port  N,  j  —  1,2 ,...,//;  thus,  R(l)^R(2)i...^R(A0  =0.  Denote  by 
R(iJ)  the  propagation  delay  along  the  control  wire  from  port  i  to  port  j.  We  assume 
that  signal  propagation  along  the  control  wire  is  slower  than  along  the  bus  and  that 
delays  along  shorter  sections  of  each  path  scale  proportionally;  i.e.,  R(l)>r  and 
R(iJ)>T(iJ)  for  all  ij. 

Specification  of  distributed  control  scheme  A1  is  in  terms  of  an  algorithm  for  an 
individual  port  j.  Packets  (for  transmission  by  port  j)  which  arrive  while  an  execution 
of  the  algorithm  by  port  j  is  in  progress  queue  externally.  Upon  completion  of  this 
execution  of  the  algorithm,  one  of  any  such  packets  immediately  becomes  available  to 
port  j  for  transmission  and  the  next  execution  of  the  algorithm  begins. 

Algorithm  A l 

*  Set  S(y)  to  1 

*  Wait  for  a  time  interval  R(J)  +  T 
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*  Wait  until  the  bus  is  observed  (by  port  j)  to  be  idle  AND  PO)  =  0;  then  start 
transmission  of  the  packet,  simultaneously  resetting  SO)  to  0. 

For  simplicity  we  assume  that  there  can  be  at  most  one  packet  in  queue  at  each 
port.  Specifically,  suppose  that  the  time  from  end  of  transmission  by  port  j  until  the 
arrival  of  a  next  packet  for  transmission  by  port  j  is  a  positive  random  variable,  AJ} 
with  finite  mean.  Also  suppose  that  the  time  for  port  j  to  transmit  a  packet  is  a 
positive  random  variable,  Lj,  with  finite  mean. 

Set 

(2.12)  W0)  =  (Wl(0 KVO), 

where  W  At)  equals  1  if  at  time  t  port  j  has  set  its  flip-flop  but  has  not  yet  completed 
the  R(J)  +  T  wait,  equals  2  if  port  j  has  completed  the  R(J)  +  T  wait  but  has  not 
started  transmission,  equals  3  if  port  j  is  transmitting,  and  equals  4  otherwise.  Next  set 

(2.13)  U(t)  =  (£/,(/) UN(t)), 

where  UjO)  equals  1  if  port  j  observes  the  bus  to  be  busy  at  time  t  and  equals  0 
otherwise.  Also  set 

(2.14)  V(t)  =  JK2il(/)fK3fi(/),r3 ,2(').F4  tl(/),....^.1(0), 

where  Vj  ^O)  equals  1  if  port  j  has  observed  that  port  k  has  set  its  flip-flop  and  equals 
0  otherwise.  (Port  j  observes  P(/)*l  at  time  t  if  and  only  if  V.At)  -  1  for  some  k<j.) 
Finally,  set 


*(/)  -  (WU),U( t),VO)). 


Then  the  stochastic  process  { A'(r) :/^0 }  is  a  gsmp  with  a  finite  state  space,  S,  and  event 
set,  £.  The  events  in  the  set  E  are:  "end  of  transmission  by  port  j,"  "end  of  wait  for 
R(J)  +  7\"  "setting  (to  1)  of  flip-flop  by  port  j,"  "observation  by  port  j  of  start  of 
transmission,"  "observation  by  port  j  of  end  of  transmission,"  "observation  by  port  j  of 
the  setting  (to  1)  of  flip-flop  by  port  k  to  the  left,"  and  "observation  by  port  j  of  the 
resetting  (to  0)  of  a  flip-flop  by  port  k  to  the  left,"  j  -  1,2 For 
s  =  (wl,...,wNtul,...,uN,v2  e  S  the  event  sets  £(j)  are  as  follows.  The  event 

set  E(s)  contains  "end  of  transmission  by  port  j"  if  and  only  if  Wj  =  3.  The  event  "end 
of  wait  for  R(j)  +  T"  £  E(s)  if  and  only  if  Wj  =  1.  The  event  "setting  of  flip-flop  by 
port  j''eE(s)  if  and  only  if  w,  =  4.  The  event  "observation  by  port  j  of  start  of 
transmission"  e  E(s)  if  and  only  if  wk  =3  for  some  k  and  w-  —  0.  The  event 
"observation  by  port  j  of  end  of  transmission"  e  £($)  if  and  only  if  wk  -  3  for  some  k 
and  Uj  =  1.  The  event  "observation  by  port  J  of  setting  of  flip-flop  by  port  k  to  the 
left"e£(r)  if  and  only  if  wk  =  1  for  some  k<j  and  vk  j  =  0.  The  event  "observation  of 
resetting  of  flip-flop  by  port  k  to  the  left"e£(r)  if  and  only  if  wk  =  3  for  some  k<j 
and  vkJ  *  1. 

The  distribution  functions  of  new  clock  times  for  events  e  e  N(s  ;s,e')  are  as 
follows.  If  e  =  "end  of  transmission  by  port  /'  e  £(r')-(£(i)-{e*})  and  /7(j';i,e*)>0, 
the  clock  setting  distribution  function  F(x;s  ,e  ,s,e)  -  P[LjZx}.  If  e  =  "end  of  wait  for 
R(J)  +  T,"  the  clock  setting  distribution  function  F(x\s' ,e  ,s,e*)  —  +  If 

e  =«  "setting  of  flip-flop  by  port  j,"  the  .  clock  setting  distribution  function 
F(x;s  ,e  ,s,e*)  =  P{Ajix\.  If  e  =>  "observation  by  port  j  of  start  of  transmission,"  the 
clock  setting  distribution  function  F(x;s',e  ,s,e‘)  =  l[7-(/c ;)  „)(•*)  If  wk  =  3-  If 
e  »  "observation  by  port  j  of  end  of  transmission,"  the  clock  setting  distribution 
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function  F(x;s',e,s,e  )  =  l[r(k  j)  «)(■*)  if  w*  -  4.  If  e  =  "observation  by  port  j  of 
setting  of  flip-flop  by  port  k  to  the  left,"  the  clock  setting  distribution  function 
F(x;s',e,s,e  )  =  ,„)(•*)  if  =  1  ( k<j ).  If  e  =  "observation  by  port  j  of 

resetting  of  flip-flop  by  port  k  to  the  left,"  the  clock  setting  distribution  function 
F(x;s  ,e  ,s,e  )  =  1  00  if  ^  =  3. 


3.  RETURNS  TO  A  FIXED  STATE 

Recurrence  properties  of  the  underlying  stochastic  process  of  a  discrete-event 
simulation  are  needed  to  establish  estimation  procedures  based  on  regenerative 
processes.  Lemma  (3,1)  is  a  special  case  of  a  generalized  Borel-Cantelli  lemma  due  to 
Doob  [2,  p.  324].  The  elementary  proof  given  below  uses  a  "geometric  trials" 
argument. 


(3.1)  LEMMA.  Let  {y^niO}  be  a  sequence  of  random  variables  defined  on  a 
probability  space  (Sl,&,P)  and  taking  on  values  in  a  set,  S.  Let  s'  e  S.  Suppose  that 
there  exists  S>0  such  that 

(3.2)  P{Yn  -j'IT,,.! . T0}  *  8  a.s. 

for  all  nil.  Then  P{Y„  =  s'  i.o.}  =  1. 


Proof:  Let  I  be  the  index  of  first  entrance  time  of  {yn:ni0}  to  state  s': 

I  =*  min  {n  i  1:  Yn  =  j'}. 


Then 


P[I  >  n]  -  m,,#/ . Y^s) 


and  it  is  sufficient  to  show  that  f,{/>n}s(l-6)n  for  ail  nil.  For  any  n. 
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PU  >  n}  =  P{Yn*s\...,Yx*s)  «  E{P{Yn+s\....Yx+s  |  Y^X.....YX}} 

mE  Virus' . . . yii} 

*  ^fy^ . yi^in-5)}  =  (l-5)/’{/>»-lj 

and  therefore  P{I>n\ l-fi)”.  □ 

Lemma  (3.1)  provides  a  means  of  showing  that  the  underlying  stochastic  process 
of  a  simulation  returns  infinitely  often  (i.o.)  to  a  fixed  state.  Specifically,  let 
{ AT(/) :/^0  j  be  a  stochastic  process  with  right-continuous  and  piece-wise  constant  sample 
paths  and  countable  state  space,  S.  Let  s' £  S  and  suppose  that  {7  :n*0}  is  an 
increasing  sequence  of  finite  (rn<»  a.s.)  state  transition  times  for  { A'C/) :/^0 }  such  that 

PIX(T„ )  =  s  i  S  a.s. 

for  some  5>0.  Then  P[X{Tn)  =  s'  i.o.}  =  1  by  Lemma  (3.1)  (with  Yn  =  X(Tn)).  In 
practice,  it  can  be  difficult  to  show  that  Tn<<*>  a.s. 

The  argument  used  in  Example  (3.3)  is  due  to  Richard  Tweedie. 

(3.3)  EXAMPLE.  In  the  token  ring  model  of  Example  (2.7)  let  Tn  be  the  nth  time  at 
which  port  1  observes  the  token,  n^O.  Then  there  is  a  packet  queued  for  transmission  at 
ports  2,3,...,fV  and  port  1  starts  transmission  of  a  packet  at  time  Tn  if  X(Tn )  =  s',  where 
s'  =»  (0,1,. ..,1,1,2).  Lemma  (3.1)  implies  that  P{X(Tn)  =  s'  i.o.}  =  1  provided  that 

(3.4)  P[Aj  >  x  +  y  \Aj  >  y}  5  P{Aj  >  x} 


for  all  xj-aO  and 
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P{Aj  £  Rj  +  ...  +  >  0, 


j  =®  1,2,. ..,N.  First  observe  that  T  <*  a.s.  since 


^Tn-Tn_ i}  *Rl  +  ...  +  RM+  2  *lLj}< 

J=  1 


for  all  nal.  Now  set 


5=1]  PlAj*  Rj  + ...  + Rn}. 
J-l 


By  Equation  (3.5),  <5>0  and  we  claim  that 


m(r„)  =  *'  |A'(rn.j) . X(T0)\  i  S. 


To  see  this,  let  Tn(J )  be  the  first  time  after  Tn  j  that  the  token  leaves  port  j;  i.e.. 


Tn(N)  «  inf  {/  >  Tn_y  N(t)  =  1  and  A/(r)  =  0} 


rn(y)  =  inf  {/  >  Tn_ A'(f)  =  y  +  1  and  M{t)  =  0}, 


y  =  1,2 . N-l.  The  definition  of  rn(y)  implies  that  there  is  no  packet  queued  for 

transmission  at  port  j  at  time  Tn(J)  and  that  Tn-Tn(J)*Rj  +  •••  +  Ry,  the  time  for  the 
token  to  propagate  from  port  y  to  port  N.  Equation  (3.4)  ensures  that 


P{Zj{Tn-)  -  1  U(rfl.1),...,Af(70)}  *  P{Aj  s  Rj  +  ...  + 


for  all  j  and  therefore  that 


P[X(T„)  «  s'\X(T^) . X{T0))  -  P{Z^Tn~)  -  1 . ZN(Tn-)  =  1  lOT,.,) . *(T0)}  2  5. 


4.  REGENERATIVE  GENERALIZED  SEMI-MARKOV  PROCESSES 


Heuristically,  a  regenerative  stochastic  process  has  the  characteristic  property  that 
there  exist  random  time  points,  referred  to  as  regeneration  points  or  regeneration  times, 
at  which  the  process  probabilistically  restarts.  Typically,  a  regenerative  process 
probabilistically  starts  afresh  when  the  process  returns  to  some  fixed  state.  The 
essence  of  regeneration  is  that  between  any  two  successive  regeneration  points  the 
evolution  of  the  process  is  a  probabilistic  replica  of  the  process  between  any  other  such 
pair  of  regeneration  points. 

In  the  presence  of  certain  regularity  conditions,  a  regenerative  stochastic  process 
{ AT(/)  :r^0 }  has  a  limiting  distribution  provided  that  the  time  between  regeneration 
points  is  finite.  Furthermore,  the  regenerative  structure  ensures  that  the  behavior  of 
the  process  between  two  successive  regeneration  points  determines  the  limiting 
distribution  of  the  process  as  a  ratio  of  expected  values.  A  consequence  of  these  results 
(Crane  and  Iglehart  [1])  is  that  a  strongly  consistent  point  estimate  and  asymptotically 
valid  confidence  interval  for  the  expected  value  of  a  general  (measurable)  function  of 
the  limiting  random  variable  X  can  be  obtained  by  observation  of  a  finite  portion  of  a 
single  sample  path  of  the  regenerative  process.  This  is  accomplished  by  simulating  the 
process  in  cycles  and  measuring  certain  quantities  defined  within  the  individual  cycles. 

Irreducible  and  positive  recurrent  continuous  time  Markov  chains  having  a  finite 
or  countable  state  space  are  the  most  familiar. examples  of  a  regenerative  process  in 
continuous  time.  The  successive  entrances  of  such  a  Markov  chain  to  any  fixed  state 
form  a  sequence  of  regeneration  points.  It  is  frequently  difficult,  however,  to  show 
that  the  underlying  stochastic  process  of  a  non-Markovian  system  is  regenerative. 


k. 
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Typically,  the  problem  lies  primarily  in  establishing  conditions  under  which  the  process 
returns  infinitely  often  to  a  fixed  state. 

The  usual  formal  definition  (cf.  Smith  [13])  of  a  regenerative  process  is  in  terms 
of  the  pasting  together  of  so-called  "tours."  We  give  an  equivalent  definition. 

(4.1)  DEFINITION.  A  stopping  time  for  a  stochastic  process  {^f(t):tkO}  is  a  random 
variable  T  (taking  values  in  [0,ao))  such  that  for  every  finite  tkO,  the  occurrence  or 
non-occurrence  of  the  event  {Ts/}  can  be  determined  from  the  history  {AT(w):u£/}  of 
the  process  up  to  time  t. 

(4.2)  DEFINITION.  The  real  (possibly  vector-valued)  stochastic  process  {Jf(/):/20}  is 
a  regenerative  process  in  continuous  lime  provided  that: 

(i)  there  exists  a  sequence  of  stopping  times  {Tk:kz 0}  such  that  { Tk  +  l-Tk:ki.O }  are 
independent  and  identically  distributed; 

(ii)  for  every  sequence  of  times  (mal)  and  kz 0,  the  random  vectors 

{X(t j),...^f(/m)}  and  {X(Tk  +  t^),...,X(Tk  +  tm)}  have  the  same  distribution  and 
the  processes  { X(t):t<Tk }  and  {X(Tk  +  /):**0}  are  independent. 

According  to  Definition  (4.2),  every  regenerative  process  has  an  embedded 
renewal  process.  The  random  times  {Tk:kzQ}  are  regeneration  points  for  the  process 
and  the  time  interval  [TkVTk)  is  called  the  /cth  cycle  of  the  process.  The 
requirement  that  the  regeneration  points  be  stopping  times  means  that  for  any  fixed  t 
the  occurrence  of  a  regeneration  point  prior  to  time  t  (i.e.,  TjS/)  depends  on  the 
evolution  of  the  process  {XXO :/£()}  in  the  interval  (0,/]  but  not  beyond  time  t. 


; 
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Proposition  (4.3)  gives  a  set  of  conditions  on  the  building  blocks  of  a  gsmp  which 
ensures  that  the  process  is  regenerative  and  that  the  expected  time  between 
regeneration  points  is  finite.  The  latter  result  is  due  to  Peter  Glynn. 

(4.3)  PROPOSITION.  Let  { AT(/) :/^0 }  be  a  gsmp  with  a  finite  state  space,  S,  and  event 
set,  E.  Suppose  that  there  exists  a  state  5q£S  and  an  event  e'e£  such  that 
P(5o;j0*ff*)>0  for  some  sQe  S  and  O(fQ;r0,e*)  =  £(jo)n  (£(*0)-{e*})  =  <p  for  all  joeS. 
Also  suppose  that  there  exists  an  increasing  sequence  of  stopping  times  {r„:n£0}  that 
are  finite  (Tn<a o  a.s.)  state  transition  times  at  which  e*  is  the  trigger  event  and  6>0 
such  that 

P{X(Tn)  =s'0\X(Tn^) . X(T0)\  i  S  a.s. 

Then  {^(/):/i0}  is  a  regenerative  process  in  continuous  time.  Moreover,  if 

_  T 

lim  — P-  =  a  <  ao  a.s. 
n 

then  the  expected  time  between  regeneration  points  is  finite. 

Proof:  Since  T„<»  a.s.  and  P{X(Tn)  -  s'0 1  AT(r/|.l),...^:(r0)}i5>0,  Lemma  (3.1) 

ensures  that  {^f(rn):n20}  hits  state  jg  infinitely  often  with  probability  one. 
Furthermore  at  such  a  time,  Tn,  the  only  clocks  that  are  active  have  just  been  set  since 
O(jQ;j0,e*)  =»  <f>  for  all  jQe  S'.  The  joint  distribution  of  X(Tn)  and  the  clocks  set  at  time 
Tn  depends  on  the  past  history  of  {X(/):/zO}  only  through  jg,  the  previous  state  s0,  and 
the  trigger  event  e  .  But  since  S  is  finite  some  previous  state,  r0,  occurs  infinitely 
often.  Therefore,  the  subset  of  times  Tn  at  which  event  t  triggers  a  transition  from 
this  state  j0  to  state  jgare  regeneration  points  for  the  process 
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To  show  that  the  expected  time  between  regeneration  points  is  finite,  let 
be  the  regeneration  points;  i.e., 

5;  =  inf  {Tk  >  S'^:  X(Tk)  =  4  X(Tk-)  =  4. 


Then  E{S'n-S'n^}<oo  if  and  only  if 

S' 

lim  —  <  00  a.s.. 


Next  observe  that  S'n  =  Tfc(n)  for  some  sequence  {fc(n):n2l}  and  that 


n 


THn ) 
kin) 


kin) 

n 


Thus, 


lim 

n-^oo 


£  a  lim 


kin) 

n 


and  it  can  be  shown  (using  an  argument  similar  to  that  in  Lemma  (3.1)  and  the 
Borel-Cantelli  lemma)  that 


lim 

n 


kin) 


<  *  a.s. 


so  that 


lim  <  »  a.s. 
-  n 

n  00 


and  E{S'„-S’n^}<*>.  □ 

(4.4)  EXAMPLE.  In  the  token  ring  model  of  Example  (2.7),  take  sQ  -  (0,1 . 1.1,2) 

and  t  -  "observation  of  token."  A  transition  to  state  ^  can  occur  when  event  e  is 
the  trigger  event  only  if  e  occurs  in  state  Jq  «  (1,..., 1,0,1)  and  in  this  case  the  set 


0(jQ;jo,e*)  =  <t> •  If  Tn  is  the  nth  time  that  port  1  observes  the  token,  there  exists  5>0 
such  that  P{X(Tn )  =  5q  |  A'(7'n),...,AT(7'0)}2:5  by  the  argument  in  Example  (3.3).  The 
successive  times  Tn  at  which  e  is  the  trigger  event  in  state  50  (and  there  is  a  transition 
to  state  Jq)  are  regeneration  points  for  the  process  {■£■(/)  :/a0}. 


L 


Next  observe  that 


Tn~  Tn- 1  *  +  —  +  Rh  +  f-i-  +  ...  +  L 


N  f  Mn 


~Nn, 


where  the  L y  are  independent  and  Ljn  is  distributed  as  Ly  Thus, 

T. 


*  y  uk 

km  t 


where  Uk  —  +  ...  +  RN  +  Llk  +  ...  +  LNk.  By  the  strong  law  of  large  numbers 


£  -  eiu)  <  -■ 

k- 1 


Therefore, 


-  T 

lim  <  «  a.s. 

n-m  n 


and  the  expected  time  between  regeneration  points  is  finite. 


5.  CONCLUDING  REMARKS 

Most  discrete-event  simulations  can  be  modeled  within  the  gsmp  framework.  In 
some  stochastic  systems,  however,  it  is  possible  to  define  a  system  state  which 
maintains  sufficient  information  to  determine  the  quantities  of  interest  and  to  specify 
an  algorithm  for  generating  sample  paths  of  the  associated  stochastic  process,  but  the 
process  does  not  have  a  gsmp  representation.  As  an  example,  suppose  that  the  state  of 


the  collision-free  bus  network  at  time  /  is  defined  to  be 


*(/)  =  (W'(fU>  (/).r(0). 

where  W(t)  and  U(t)  are  as  in  Example  (2.11),  F-(/)  is  the  number  of  ports  to  the  left 
observed  by  port  j  to  have  set  their  flip-flop  and  V(t)  =  ( The  process 
{A(/):/2:0}  has  a  finite  state  space,  S.  It  does  not  appear  to  be  possible,  however,  to 
specify  an  event  set  £  such  that  {AT(/):/aO}  is  a  gsmp  with  state  space,  S,  and  event  set, 
E.  For  example,  suppose  that  E  is  the  set  of  events:  "end  of  transmission  by  port  j," 
"end  of  a  wait  for  R(J)  +  T,"  "setting  of  flip-flop  by  port  j,"  "observation  by  port  j  of 
start  of  transmission,"  "observation  by  port  j  of  end  of  transmission,"  "observation  by 
port  j  of  setting  of  flip-flop  by  port  to  the  left,"  and  "observation  by  port  j  of  resetting 
of  flip-flop  by  port  to  the  left,"  j  =  1,2,..., jV.  Then  {A'(/):*£0}  fails  to  be  a  gsmp 
because  there  are  states  for  which  it  is  not  possible  to  determine  whether  or  not  the 
event  "observation  by  port  j  of  setting  of  flip-flop  by  port  to  the  left"  or  the  event 
"observation  by  port  k  of  resetting  of  a  flip-flop  by  port  to  the  left"  is  active.  (Select 
ij,  and  k  such  that  1  zi<j<kzN  and  take  j  =  ( )  with  =  3, 
wk  —  1,  and  Vj  =  1.) 
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